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ABSTRACT 



Current educational theory and practice clearly show that 
project -based instruction has the potential to enhance learning. Preliminary 
findings on one type of project -based learning in which students take on the 
role of hypermedia designers support this claim. This study examined the 
effect of being hypermedia designers on fourth-graders' motivation and 
learning of design knowledge. Both quantitative (motivation scale, design 
questionnaire, task ranking, HyperStudio tests, and analysis of 
student -created programs) and qualitative data (observations, response log 
entries, and interview) were collected. The findings showed that engaging 
students in hypermedia authoring could enhance their motivation, and allowing 
students to be hypermedia designers could support the development of design 
knowledge and higher order thinking skills. The skills most affected in this 
study included planning, presentation, reflection, collaboration, task 
distribution, and time management. Three tables presents data on: motivation, 
including measures of challenge, curiosity, mastery, judgment, and criteria; 
design skills, including categories of audience, presenting, planning, 
interest, mental effort, and collaboration; and the importance of the design 
tasks ranked by the students. Contains 21 references. (Author/DLS) 
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Abstract: Current educational theory and practice clearly show that project-based 

instruction has the potential to enhance learning. Preliminary findings on one type of 
project-based learning in which students take on the role of hypermedia designers support 
this claim. This study examined the effect of being hypermedia designers on fourth-graders’ 
motivation and learning of design knowledge. The findings showed that engaging students 
in hypermedia authoring could enhance their motivation, and allowing students to be 
hypermedia designers could support the development of design knowledge and higher order 
thinking skills. The skills mostly affected in this study included planning, presentation, 
reflection, collaboration, task distribution, and time management. 
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1. Research Framework 
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Project-based learning, requiring students’ active participation and engaging them in authentic 
problem investigations, is considered to have great potential to enhance students' motivation and learning 
[Blumenfeld, Soloway, Marx, Krajcik, Guzdial, & Palincsar 1991]. Technology can play an important role in 
facilitating project-based learning by enhancing students' interest and supporting information gathering and 
presentation [Blumenfeld, et. al. 1991]. Engaging students in hypermedia/multimedia design is one type of 
project-based learning which has shown some encouraging results in promoting higher order thinking skills. 

Engaging learners as hypermedia designers is an instructional strategy that invites learners to become 
intellectual partners with the technology and provides them an opportunity of using the technology as a 
cognitive tool to extend their minds and to construct their own knowledge [Jonassen 1994], [Solomon, 
Perkins, & Globerson 1991]. Perkins [Perkins 1986] states that the process of design promotes learners' 
active pursuit and use of knowledge. As designers, learners are encouraged to be creative, to integrate new 
knowledge with their prior knowledge, and to pursue their own intended goals actively. Because designing 
hypermedia programs "taps a diverse set of skills” [Carver et al., p. 388 1992] and involves both the process 
and product, educators believe it can provide a concrete and meaningful context for developing higher order 
thinking skills. 

Research on engaging learners as hypermedia designers has only appeared recently. Lehrer, 
Erickson, and Connell [Lehrer, Erickson, and Connell 1994] conducted a study in which ninth-graders created 
hypermedia presentations on American history for other students. They found that students' time on-task 
increased significantly over the course of successive design projects. In addition, the study showed that the 
design process helped students to internalize various design skills. Students reported increases in mental 
effort and involvement, interest, planning, collaboration and individualization. Supporting their findings, 
Beichner [Beichner 1994] found in his study that junior high school students were highly motivated and often 
spent extra time when working on producing a multimedia program. Spoehr's study [Spoehr 1994] showed 
that designing hypermedia programs could help students develop more complex knowledge representations 
and assist the development of their thinking skills. A study by Liu and Rutledge [Liu & Rutledge 1997] 
found that high school students showed a significant growth in their value of intrinsic goals, and hypermedia 
design helped them to acquire several critical design skills. Other studies have shown that seventh graders, 
both advanced and behavioral ly/emotionally disordered, were motivated by creating multimedia projects 
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[Orey, Fan, Scott, Thuma, Robertshaw, Hogle, & Tzeng 1997]; both boys and girls appeared to like doing 
multimedia projects [McGrath, Cumaranatunge, Ji, Chen, Broce, & Wright 1997]; and building a community 
of designers in the classroom could foster learning for students and teachers [Erickson & Wilhelm 1996]. 

The studies on learner-as-hypermedia/multimedia-designers suggest the following: 1) such a learning 
environment can have a positive impact on students' motivation toward learning; 2) such an environment 
encourages creativity and enhances the development of cognitive skills; and 3) high and middle school 
students learned design skills in addition to content and computer knowledge. While the preliminary findings 
in this area have shown some encouraging results, much is to be learned about designing and implementing 
such a learning environment effectively for different learners and curriculum needs. 



2. Purpose Of The Study 

Given the preliminary findings of the positive effects of the "learner as a hypermedia designer 
environment" on high and middle school students, we are interested in finding out if elementary school 
students can benefit from a similar learning environment, and, if so, what issues and factors need to be 
considered in designing such a learning environment. The research question for this study, therefore, was 
"What is the effect of being hypermedia designers on elementary school students' motivation and learning of 
design knowledge?" 



3. Design Of The Study 

3.1 Participants 

The participants of this study were two intact fourth-grade classes (N= 38) from an elementary 
school in a mid-size city in the southwestern United States. Seventy-nine percent of the student population 
were white, and twenty-one percent were minority. Thirteen percent (N= 5) were identified as talented and 
gifted students (TAG). Twenty-four percent (N=9) were resource students who need additional academic 
help in various subject matters. Seventeen were female and twenty-one were male students. 



3.2 The Hypermedia Learning Environment 

The research study took place over two semesters, from November 1996 to April 1997. Both classes 
participated in this hypermedia authoring project as part of their daily 50-minute science class four days a 
week. The entire project consisted of three phases. During the first phase, which lasted approximately six 
weeks, students in both classes received the same instruction on the use of HyperStudio , an open-ended 
authoring system specially designed for children to create hypermedia programs with links, nodes, colors, 
text, graphics, animation, sound and video. The purpose of this phase was for students to acquire the technical 
skills and create a small HyperStudio stack with guidance. 

In phase II of the project, which lasted approximately four and half weeks, students in one class 
worked in teams in a leamer-as-designer environment, while students in the non-designer class worked 
independently. Both classes created HyperStudio stacks as part of a unit of study. In the non-designer class, 
the typical sequence for a day was for the teacher to spend about 15 minutes introducing the science content 
and then instruct students to create a card in their HyperStudio stacks on this topic. While students in the 
non-designer class made decisions individually about the content and organization of their stacks, students in 
the designer class worked in groups and made those decisions collaboratively. Because students in the 
designer class shared the responsibility for the creation of a group stack, individual students created fewer 
cards in the designer class than in the non-designer class. However, students in the designer class received 
instruction in the design process, including instruction on planning, division of responsibility, peer evaluation, 
and interface design principles. In a typical week , students spent about thirty percent of their time on of the 
were class-related activities, such as content teaching, design discussion, or group presentation, while about 






seventy percent of the activities were group related and led by students, such as planning, research, task 
distribution, and the creation of cards, stacks and multiple types of media. Collaboration within the group 
was required and between the groups was strongly encouraged. 

Phase III of the project was similar to phase II, with the exception that, for both groups, the support 
from the teacher and researchers was faded. 



3.3 Data Sources 

Both quantitative (motivation scale, design questionnaire, task ranking, HyperStudio tests, and 
analysis of student-created programs) and qualitative data (observations, response log entries, and interviews) 
were collected. The triangulation of the multiple data sources provides a better picture of the learning 
environment under study. 

Harter's scale of intrinsic versus extrinsic orientation in the classroom was used [Harter 1980]. It is 
30-item scale that measures five aspects of motivation: (1) preference for challenge vs. preference for easy 
assignments; (2) curiosity/interest vs. pleasing the teacher/getting grades; (3) independent mastery vs. 
dependence on the teacher; (4) independent judgment vs. reliance on the teacher's judgment; and (5) internal 
criteria for success/failure vs. external criteria. This instrument was developed for use in grades three through 
six, and the reliability index for the subscales range from .68 to .84 [Harter 1980]. It was given to both 
classes at the beginning and at the end of the hypermedia project. 

The design questionnaire was based on one developed by Lehrer and his colleagues to assess 
learners' development of various higher order thinking skills needed in producing hypermedia projects 
[Lehrer, et al., 1994]. This project questionnaire has been used in the research of leamer-as-hypermedia- 
designer with a reliability index of .97 [Carver et al. 1992], [Lehrer, Erickson, & Connell 1993], [Lehrer, 
Erickson, & Connell 1993], [Liu, in press], [Liu & Rutledge 1997]. This self-report questionnaire was 
adapted to include only statements directly linked to the project. The KR 20 for this modified version was 
.82. Students in both classes completed the instrument at the end of the project. 

In the task ranking questionnaire, students were given a list of eight tasks relevant to their project 
development, and were asked to rank the tasks according to their relative importance. This instrument was a 
simplified version, tailored to the fourth graders, of the task ranking evaluation used in a number of research 
studies on the same topic [Lehrer et al. 1994], [Liu & Rutledge 1997], [Liu in press]. Students were given 
this instrument at the end of the project. 

Performance was assessed using the stacks created during phase III. These stacks were judged by 
two external evaluators in terms of the product quality using a fourteen item survey, with each item scored on 
a five-point scale. The total score for the survey was the composite score from the two raters, ranging from 
28 (14 items X 1 minimum score for each item X 2 raters) to 140 (14 items X 5 maximum score for each item 
X 2 raters). The development of this evaluation was based upon the literature on assessing interactive 
multimedia/hypermedia programs [Fleming & Levie 1993], [Laurel 1993], [Mullet & Sano 1995]. It 
addressed the application of four design principles: consistency, contrast, legibility and simplicity. The 
interrater reliability of all items for all the stacks was 88%. 

Interviews were conducted with the teacher (at the mid-point of the project), the parents (during two 
open houses), and the students (at the end of the project). Students selected for interviews were from both 
classes and from various ability groups. Interview questions focused on perception of motivation and 
comparison with other types of projects. 

The teacher kept a journal detailing her observations of the classroom, and students were assigned 
three response log entries at different times. Observations and response log entries focused on student 
engagement, their perception of learning, and reflections on the design process. 



3.4 Analysis 

To understand if hypermedia design had an impact on the elementary school students' motivation, 
five two-factor mixed ANOVAs were run with the grouping (designer vs. non-designer) as the between- 
subjects independent variable and the data collection points (pre vs. post) as the repeated measure 
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independent variable. The dependent variable was the pre and post scores of each of the five measures in 
Harter’s motivation scale (challenge, curiosity, mastery, judgment, and criteria). Unpaired t-tests were 
conducted between the designer and non-designer groups on each of the five categories of the project design 
questionnaire (audience, presenting, planning, interest, mental effort, and collaboration). Students’ responses 
to the design task ranking were analyzed descriptively. An unpaired t-test between the designer and non- 
designer groups was conducted on students’ performance in creating their stacks. 

The interview data and the response logs were analyzed using a two-level scheme following the 
guidelines by Miles and Huberman [Miles & Huberman 1994]. At the first level, codes were generated 
directly from the interviews and response logs through multiple passes of the data examination. At the second 
level, codes generated directly from the data were regrouped into more general categories. Patterns from the 
data were extracted and the relationships between the coded segments were compared and contrasted. With 
the research question as a guide, the data were then sorted into categories and sub-categories according to 
their common themes and shared relationships. The observation data were used to corroborate the findings 
from the interviews and response logs. 



4. Results 

4.1 Results from the Quantitative Data 

For the motivation scale, the results of the two-factor mixed ANOVAs indicated that there was a 
significant two-way interaction between the grouping (designer vs. non-designer) and the data collection 
points (pre vs. post) for scores on judgment: F(l,35) = 5.3, p <.05 [see Tab. 1]. The designer group increased 
its score on judgment from pre to post whereas the non-designer group decreased its score slightly. That is, 
the difference between the pretest and posttest for the designer group was significantly greater than that for 
the non-designer group. The two groups, however, were not significantly different in scores of challenge, 
curiosity, mastery and criteria: F(l,35) c hallenge = 02, p=.88; / r (l,35) C uriosity = 102, p=.32; FXl^jmastery 
= 1.11, p=.30; F(l,35)criteria = 02, p=.88 [see Tab. 1]. 



Challenge Curiosity Mastery Judgment Criteria 





Pre 


Post 


Pre 


Post Pre 


Post 


Pre 


Post 


Pre 


Post 




Designer 


2.7 


2.6 


2.83.0 


2.9 


2.8 


2.6 


2.9 


2.4 


2.3 




Group 

(N=19) 


(-7) 


(.7) 


(.6) 


(.6) 


(.7) 


(.6) 


(.9) 


(.7) 


(.9) 


(.7) 


Non-designer 




2.9 


2.93.0 


2.9 


2.7 


2.8 


2.5 


2.3* 


2.6 


2.5 


Group 

(N=18J 


(.8) 


(.8) 


(.8) 


(.8) 


(.7) 


(.8) 


(.7) 


(.7) 


(.7) 


(.9) 



* = significant two-way interaction at p < .05 



Table 1: Mean and Standard Deviation (in Parenthesis) for the Motivation 



For the design questionnaire, the unpaired-T tests showed that there were significant differences 
between the groups for the following categories: (1) planning: t( 1 ,35) = 3.0, p <.01; (2) interest: t( 1 ,35) = - 
2.8, p <.01; and (3) collaboration: t(l ,35) = 2.2, p <.05 [See Tab. 2]. The average scores for planning and 
collaboration were significantly higher for the designer group than for the non-designer group. However, the 
non-designer group had significantly higher scores in the category of interest than those for the designer 
group. Such results showed that the students in the designer group had a better understanding of the 
importance of planning and collaboration, while students in the non-designer group showed more interest in 



the project than the designer group. No significant differences were found between the groups for the 
categories of audience, presentation, and mental effort [see Tab. 2]. 
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Audience 


Presenting 


Planning 


Interest 


Mental 

Effort 


Collaboration 


Designer 3.0 


3.4 


4.4* 


3.9 


2.8 


3.4* 


Group (1.1) 

(N=19) 


(.6) 


(.8) 


(.6) 


(1.0) 


(1.3) 


NonDesigner 3.1 


3.4 


3.6 


4.5** 


3.1 


2.6 


Group (.9) 

(N=18) 


(.8) 


(.8) 


(.3) 


(1.1) 


(1.0) 



* significantly different from the non-designer group, P < .05. 

** significantly different from the designer group, P < .05. 

Table 2: Mean and Standard Deviation (in Parenthesis) for the Design Skills 

In the task ranking measure, students from both classes were given a list of eight design tasks and 
were asked to rank their relative importance. Both the designer and non-designer groups ranked planning 
tasks as most important [see Tab. 3]. However, the two groups were different in ranking some of the other 
tasks. The designer group ranked the task of having "someone to try out the stack" as having greater 
importance than the non-designer group did. It ranked the tasks of "making the graphics very colorful," and 
"making animation" less important than the non-designer group did. Some of these differences between the 
two groups were statistically significant [see Tab. 3]. 

Planning and having others evaluate a product are more important steps for hypermedia design than 
making graphics colorful or making animation [Lehrer et al. 1994], [Liu & Rutledge 1997]. The results of the 
task ranking showed that the designer group had an overall better understanding of the design tasks and their 
relative importance than the non-designer group. 



Design Tasks 


Designer 

Class 

Average** 


NonDesigner 

Class 

Average** 


1. Plan and write down on your index card what your card will look like. 


1.9 


1.7 


2. Discuss with your group what information to include. 


2.0 


2.6 


3. Make sure the buttons and colors are consistent from one card to another. 


4.0 


4.8 


4. Someone to try out the stack. 


4.8 


6.6* 


5. Make the graphics very colorful. 


5.7 


4.7 


6. Scan in pictures. 


5.3 


5.1 


7. Make sounds. 


6.1 


5.9 


8. Make animation. 


6.1 


4.5* 



* significantly different from the designer group, p < .01 according to the unpaired-tests. 

** The lower the mean, the higher the importance. The boldface letters indicate the differences in ranking 
between the two groups. 

Table 3: Importance of the Design Tasks Ranked by the Students 



Performance on the stacks created during phase III was assessed by two external evaluators with 
regard to the quality, the breadth, the depth and the use of the four design principles (consistency, contrast, 
legibility and simplicity). The unpaired t-test showed that there were significant differences between the 
designer and non-designer groups: t (1,19) = 2.6, p <.01 with the designer group having a mean of 95.2 and 
the non-designer group having a mean of 74.94. The higher the mean, the better the quality of the stacks. 
The designer group contributed 5 stacks (one from each of the 5 groups) and the non-designer group 
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contributed 16 stacks (one from each student). The highest score for the designer group was 120 and the 
lowest score was 63, whereas the highest score for the non-designer group was 93 and the lowest score was 
50. Eight of the 14 statements in the stack evaluation survey relate directly to the application of the four 
design principles, including "The text on the screen is very readable,” and "The navigation buttons are laid 
out consistently on each card," The designer group did significantly better than the non-designer group when 
their stacks were evaluated using these eight statements: Mean designer = 61, Mean ncm designer =45.38, 
t( 1 , 1 9) = 4.25, p < .01. These findings suggested that on the whole, the quality of the stacks produced by the 
designer group was superior to those produced by the non-designer group. 



4.2 Results from the Qualitative Data 

The interviews with the students and their entries in their response logs showed that both the designer 
and non-designer groups enjoyed the project and that they felt this high interest level impacted their level of 
learning. Parents seemed to agree that the project had motivated their children and made learning more fun. 
Their comments included "It really grabbed the kids' attention" and "K was so excited, motivated, and proud 
to have created and succeeded in this project." The teacher shared this view on students' interest, explaining 
that “They look forward to coming to the computer lab to continue their projects." 

The interview data and response logs indicated that students in both classes learned design skills by 
participating in this project. Students in both groups mentioned that they learned the technical skills of how to 
scan pictures, make colorful backgrounds, and use clipart pictures to enhance their stacks. Additionally, 
several students in the designer group mentioned the value of collaboration: "I'm getting a learning experience 
working in groups and deciding what's best for the group," Students from both groups appeared to recognize 
the importance of planning, though responses from the designer group were more concrete than those by the 
non-designer group. For example, students in the designer group commented on the need for planning in 
order to ensure consistency and avoid redundancy. One parent of a student in the designer group summarized 
the feelings of many parents well: "What a wonderful application and assimilation of learned content as well 
as skills. Clearly, the computer skills are of great value, but it is really exciting to see the children apply 
development skills and make choices for format and presentation." 

The teacher commented on how a few students who were identified as having difficulty in learning 
reacted to the different treatments, stating that in the beginning those in the non-designer group seemed to 
"accept more responsibility for their work as it is [an] individual [project]," while the low ability students in 
the designer class tended to depend on the stronger students in the group. However, during the course of the 
project, the low ability students in the designer group eventually took responsibility for their group's work 
with the help of their team members. In the end, "the needy students did more [for their stacks] in the 
designer group than those in the non-designer group because of the continual help from other members." 
while those low ability students in the non-designer group who did not ask for help and did not work hard 
accomplished little. In commenting about the gifted children, the teacher said that "They generally would do 
well whenever given the opportunity. This [the project] is a vehicle for them doing that." The gifted students 
in the designer group took initiative to help others in the groups and ensure that work was completed on time. 



5. Discussion 

Using HyperStudio as the students in the non-designer group did is a typical way of using 
hypermedia authoring in a classroom. Engaging students as hypermedia designers, on the other hand, extends 
hypermedia authoring by placing students in a designer position. Instead of merely learning the technical 
skills and creating a project, the designers need to consider other issues such as the needs of the audience, the 
distribution of work in a group, the management of time and resources, and the deadline. They need to 
implement steps such as planning, designing, evaluation, and discussion. The authenticity and complexity of 
the design tasks provide students a learning environment where they can develop higher order thinking skills 
and skills of high value to the work place. This study investigated the effect of being hypermedia designers 
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on elementary school students' motivation and learning of design knowledge, and found some interesting 
results. 

The findings on motivation, as reflected in the Harter's motivation scale [Harter 1980], showed that 
there were no significant differences between the designer and non-designer groups on the categories of 
challenge, curiosity, mastery and criteria. That is, both groups were interested in the hypermedia project, a 
finding which was supported by the qualitative data in that comments from students in both groups, their 
parents, and the teacher showed that the students enjoyed doing the project and viewed as a good way to learn 
science as well as computer skills. This finding is consistent with the literature in showing that engaging 
students hypermedia authoring is motivational and promotes students’ interest in learning [Beichner 1994], 
[McGrath et al. 1997], [Orey et al. 1997]. 

The designer group, however, outperformed the non-designer group on the category of judgment 
from pre to post, indicating that they felt more comfortable making independent judgments, rather than 
relying on the teacher's judgment, than their peers in the non-designer group. This suggests that the leamer- 
as-designer environment may encourage the development of intrinsic motivation. 

Both the Harter's motivation scale and the qualitative data showed that all students liked and enjoyed 
the project. Yet, the results on the project questionnaire showed that students in the non-designer group 
showed more interest than students in the designer group in the project. This gap between the two groups can 
possibly be explained by the frustrations experienced by the designer group, who had a greater variety of 
activities to handle, and who were responsible for planning and providing peer support. The peer pressure 
created by heterogeneous grouping, while beneficial, may also have added to the frustration. Finally, the 
designer group also had to transfer cards from one disk to another to create a group stack, a fact which 
became frustrating as file sizes grew and work was lost due to computer error. 

Learning the science content was one of the objectives of the project. The two science tests (on 
plants and oceans) showed that students in both groups had significantly increased their knowledge on the 

subject matter (p < .01) from the pretest to the posttest In addition to the science knowledge which the 
students would have gained by working on other types of projects, this project allowed the students to learn 
some important technical skills involved in hypermedia authoring. 

For the designer group, students also learned skills such as planning, project management, reflection 
(getting feedback from the peers and the audience), and working in groups to achieve a common goal. The 
statistical analyses indicated that students who were hypermedia designers had a significantly better 
understanding of planning and collaboration than the non-designer group, and valued these tasks above those 
of a more mechanical nature, a finding which supports other research [Lehrer et al. 1994], [Liu in press], [Liu 
& Rutledge 1997]. As Carver and her colleagues pointed out [Carver et al. 1992], such skills are invaluable 
for students to learn, especially when they are young. It is obvious from the data that the students as well as 
their parents viewed the opportunity of learning both science and hypermedia design and authoring as not 
only important but also beneficial to their future. 



6. Conclusion 

In an effort to examine if being hypermedia designers can have an effect on elementary school 
students' motivation and learning of design knowledge, this study found that engaging students in hypermedia 
authoring can enhance students' motivation, and allowing students to be hypermedia designers can support the 
development of design knowledge and higher order thinking skills. The hypermedia design project allowed 
the students to grow at their own pace cognitively, affectively and socially. The findings of this study support 
other research on the same topic in showing the benefits of a leamer-as-designer environment [Beichner 
1994], [Erickson & Wilhelm 1996], [Lehrer et al. 1994], [Harel 1991], [Kafai 1996], [Liu in press], [Liu & 
Rutledge 1997], [McGrath et al. 1997], [Orey, Fan, Scott, Thuma, Robertshaw, Hogle, & Tzeng 1997], 
[Spoehr 1993]. At the same time, it calls for more research in replicating and extending the findings. 
Literature has clearly shown the benefits of engaging students as hypermedia/multimedia designers. 







*The science tests are the ones the teacher had used in the science class last year and would give to students regardless of 
types of projects they were working on. 



Designing such a learning environment is nevertheless a very challenging task. Continuous research on the 
topic can advance our understanding and help us to search for new ways to design such a learning 
environment effectively. 
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